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Abstract 

The  identification  of  certain  groups  of  variables  in  optimization  problems  is  an  important 
issue  and  can  be  used  to  computational  advantage.  In  this  paper  new  logarithmic  indicators  are 
introduced.  It  is  demonstrated  that  the  logarithmic  Tapia  indicators  have  superior  ability  for 
identifying  several  subgroups  of  variables  in  the  context  of  primal-dual  interior-point  methods. 
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1  Introduction 


This  paper  concerns  the  identification  of  several  subgroups  of  variables  in  interior-point  methods 
for  linear  programming.  We  consider  the  linear  programming  problem  in  the  standard  form 

minimize  cTx 

subject  to  Ax  =  b  (1.1) 

x  >  0, 


where  c  £  Rn,  b  £  Rm,  A  £  Rmx"  <  n)  and  A  has  full  rank  m.  The  dual  problem  of  (1.1)  can 
be  stated  as 

minimize  bTy 

subject  to  At A  +  y  =  c  (1-2) 

V  >  o, 

where  A  £  Rm ,  and  y  £  Rm  are  the  Lagrange  multiplier  vectors  corresponding  to  the  equailty  and 
inequality  constraints  of  problem  (1.1)  respectively.  The  multipliers  A  are  also  known  as  the  dual 
variables  and  y  as  the  dual  slack  variables.  The  first-order  optimality  (or  Karush-Kuhn- Tucker) 
conditions  for  problem  (1.1)  are: 


and 


F(x,y,\)  = 


*  Ax  —  b  ^ 
AT\  +  y  —  c 

V  XFe  ) 


-  0 


(1.3) 


{x,y)  >  0 


(1.4) 


where  X  =  diag(x),  Y  =  diag(y)  and  e  is  the  n-vector  of  all  ones.  A  point  (x,y,X)  is  said  to  be 
strictly  feasible  if  it  satisfies  Ax  =  6,  AT A  +  y  =  c,  and  (x,y)  >  0.  A  solution  pair  ( x,y )  is  said  to 
satisfy  strict  complementarity  if  in  addition  to  complementarity  XYe  =  0,  it  satisfies  x  +  y  >  0. 

It  is  now  well  understood  that  the  primal-dual  interior-point  methods  can  be  viewed  as  damped 
and  perturbed  Newton’s  method  applied  to  the  nonlinear  system  of  equations  (1.3).  For  more 
details,  see  Kojima,  Mizuno,  and  Yoshise  [14]  and  Zhang,  Tapia,  and  Dennis  [33].  The  algorithmic 
framework  for  such  methods  is  the  following 


Algorithm  1.1  (Primal-Dual  Interior-Point  Method) 

Given  a  starting  point  ( x° ,  y°,  A0)  with  ( x° ,  y°)  >  0.  For  k  =  0, 1, . . .,  do 

1.  Choose  ok  £  (0,1)  and  set  p(x,y)  =  ok  ^  ^  y  . 

2.  Solve  the  following  system  for  (Axk,Ayk,AXk): 

F'(xk,  yk ,  Xk)(Ax,  Ay,  AX)  =  -F(xk,yk,  Xk )  +  p(xk,yk)e  (1.5) 
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3.  Choose  Tk  €  (0, 1)  and  set  the  steplength  ak  =  min(l,r*afc)  where 


min  ((Xk)-1Axk,(Yk)~1Ayk)' 

4-  Form,  the  new  iterate 

(xk+1,yk+1,Xk+1)  =  (xk,yk,Xk)  +  ak(Axk,Ayk,AXk). 

We  emphasize  that  the  algorithmic  parameters  ak  and  rk  are  under  the  control  of  the  algorithm 
designer  with  the  restriction  that  0  <  ( ak,rk )  <  1.  The  paramter  ak  is  sometimes  called  the 
centering  paramter.  Note  that  the  choice  of  steplength  ak  in  Step  3  of  Algorithm  1  guarantees 
(xk+1  ,yk+1)  >  0.  In  Step  2  e  =  (0, . . ., 0, 1, . . .,  1)T,  with  n  +  m  zero  components.  We  also 
emphasize  that  the  iteration  sequence  generated  by  Algorithm  1  is  not  necessarily  feasible. 

We  use  the  notation  T  and  S  to  denote  the  feasible  set  and  the  solution  set,  respectively,  of 
problem  (1.3).  In  many  papers  on  interior-point  methods,  the  assumption  that  T  contains  strictly 
feasible  points  is  made.  This  assumption  ensures  that  S  is  nonempty  and  compact.  For  reasons 
that  will  become  clear  in  Section  4  we  will  make  the  assumption  that  S  is  bounded  only  as  needed. 

We  use  the  following  notation 

•  Af=  {1,2,...,  n}. 

•  ri(U )  and  drU  denote  the  relative  interior  and  the  relative  boundary  of  a  given  set  U . 

•  Z  denotes  the  set  of  indices  of  variables  that  are  zero  in  every  solution  of  problem  (1.1). 

•  Z  =  Af  —  Z  denotes  the  set  of  indices  of  variables  that  are  positive  in  the  relative  interior  of 
the  solution  set  of  problem  (1.1). 

It  is  known  that  Z  U  Z  =  Af.  It  is  also  known  that  if  we  consider  the  dual  problem  (1.2),  then  Z 
and  Z  will  be  the  sets  of  indices  of  the  dual  slacks  ?/,  that  are  positive  and  zero,  respectively,  in  the 
relative  interior  of  the  solution  set  of  that  problem. 

The  identification  of  subgroups  of  variables  in  primal-dual  interior-point  algorithms  can  be  used 
in  several  ways  to  computational  advantage.  Such  applications  include,  reducing  the  dimension  of 
the  problem,  and  obtaining  a  good  initial  basis  when  crossing  over  to  a  simplex-type  method  to 
determine  a  basic  solution.  It  can  also  play  a  role  in  obtaining  highly  accurate  solutions  when 
combined  with  the  projection  method  proposed  by  Ye  [31],  perturbation  methods  proposed  by 
Mehrotra  [21]  or  successive  projection  methods  used  by  El-Bakry  [4].  Finally  this  information 
can  be  an  important  ingredient  in  building  preconditioners  for  solving  linear  systems  arising  in 
interior- point  methods,  see  Gill,  Murray,  Ponceleon,  and  Saunders  [8]. 
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Many  authors  have  contributed  to  the  goal  of  identifying  subgroups  of  variables  by  introducing 
indicator  functions  in  interior-point  methods.  Gill,  Murray,  Saunders,  Tomlin,  and  Wright  [9], 
Karmarkar  and  Ramakrishnan  [12],  McShane,  Monma  and  Shanno  [19],  Tone  [28],  Lustig,  Marsten, 
and  Shanno  [17],  Dantzig  and  Ye  [3],  and  Boggs,  Domich,  Donaldson  and  Witzgall  [1],  among  others, 
proposed  the  use  of  variables,  either  primal  or  dual,  to  predict  members  of  Z .  Tapia  [24]  introduced 
two  indicators  in  the  context  of  identifying  active  constraints  in  nonlinear  constrained  optimization 
problems.  Kojima  [13]  proposed  an  indicator  for  use  in  Karmarkar-type  algorithms.  Ye  [30]  and 
Todd  [27]  introduced  two  indicators  for  Karmarkar-type  and  primal-dual  algorithms.  Choi  and 
Goldfarb  [2]  proposed  two  indicators  similar  to  the  Todd- Ye  indicators  with  the  advantage  that 
their  indicators  can  be  used  in  algorithms  that  are  not  necessarily  interior-point  methods.  Tapia 
and  Zhang  [25]  proposed  an  indicator  that  can  be  used  in  primal,  dual,  or  primal-dual  interior- 
point  methods.  Kovacevic-Vujcic  [15]  introduced  an  indicator  that  is  superlinearly  faster  than  the 
variables  in  Karmarkar-type  methods.  The  ratio  of  primal  variables  and  dual  slacks  was  used  as 
an  indicator  by  several  researchers  including  Gay  [7],  Ye  [31],  and  Lustig  [16].  Mehrotra  [20]  used 
an  indicator  based  on  the  relative  change  in  the  dual  slack  variables.  Resende  and  Veiga  [23]  used 
the  reciprocal  of  the  dual  slack  variables  as  indicators.  Recently  de  Vreede  [29]  proposed  ????.  For 
a  thorough  study  of  indicator  functions  in  interior- point  methods  we  refer  the  reader  to  the  recent 
paper  by  El-Bakry,  Tapia,  and  Zhang  [5].  de  Vreede  [29]  performed  numerical  comparisons  between 
several  indicators.  In  all  these  papers,  the  focus  was  on  the  identification  of  the  two  basic  subgoups 
of  variables  mentioned  above,  namely  variables  that  are  zero  everywhere  in  the  solution  set  and 
variables  that  are  positive  in  the  relative  interior  of  that  set.  The  question  as  to  whether  further 
subgroups  of  variables  can  be  identified  is  the  theme  of  this  paper. 

In  El-Bakry  [4]  and  EL-Bakry,  Tapia,  and  Zhang  [5],  an  indicator  function  I(xk,yk)  is  said  to 
satisfy  both  the  strict  and  uniform  separation  properties  if 


lim  I(xk,yk) 

k — ►oo 


0!  if  i  e  ^ 

02  \uiz  ’ 


(1.6) 


where  0i  <<  02.  For  brevity  we  will  say  that  an  indicator  function  satisfies  the  0i  —  02  separation 
property  if  it  satisfies  (1.6). 

This  paper  is  organized  as  follows;  in  Section  2,  the  Tapia  indicators  are  investigated  when  the 
sequence  of  centering  parameter  {crk}  is  bounded  away  from  zero.  In  Section  3  we  introduce  the 
logarithmic  indicator  functions  and  establish  their  separation  properties.  Section  4  is  devoted  to 
numerical  experiments  with  a  subset  of  the  Netlib  [6]  set  of  linear  programming  test  problems. 
Interpretation  of  our  numerical  experimentation  is  attempted  in  Section  5.  The  logarithmic  indi¬ 
cators  for  the  dual  variables  is  considered  in  Section  6.  Final  conclusions  are  given  in  Section  7. 
Throughout  this  paper  we  consider  the  logarithm  to  be  the  natural  logarithm. 
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2  Three  Indicators 


In  this  section  we  list  three  indicators  for  identifying  zero  and  nonzero  variables  in  interior-point 
methods  for  linear  programming.  The  reasons  for  choosing  these  particular  indicators  are  that  they 
are  not  expensive  to  compute  and  that  they  do  not  require  nondegeneracy  or  (linear)  feasiblity 
assumptions.  These  indicators  are  the  variables  used  as  indicator,  the  primal-dual  indicator,  and 
the  Tapia  indicators. 

The  variables  used  as  indicator  is  probably  the  most  widely  used  indicator  in  constrained  opti¬ 
mization.  This  indicator  can  be  defined  for  both  primal  and  dual  variables 

VP{xk)  =  xk  and  VD(yk)  =  yk. 

Another  common  indicator  in  the  area  of  primal-dual  interior-point  method  is  the  primal-dual 
indicator 

PD(xk,yk)  = 

yk 

Tapia  [24]  proposed  the  following  two  indicators 

rk+ 1  k+i 

?>(**)  =  —  and  W)  =  L 

The  variables  as  indicator  do  not  satisfy  the  uniform  separation  property  (1.6).  On  the  other 
hand  both  the  primal-dual  and  the  Tapia  indicators  satisfy  this  property  with 

•  9\  =  0  and  02  =  +00  for  the  primal-dual  indicator. 

•  Oi  =  0  and  02  =  1  fot  the  Tapia  indicators. 

This  0-1  separation  property  of  the  Tapia  indicators  can  be  obtained  when  the  sequence  of  centering 
paramters  {(Tk}  is  chosen  to  converge  to  zero,  see  El-Bakry,  Tapia,  and  Zhang  [5].  An  example 
that  shows  the  behavior  of  the  primal  Tapia  indicator  for  ADLITTLE,  one  of  the  Netlib  set  of 
test  problems,  is  given  in  Figure  2.  Observe  that  the  trajectories  of  the  Tapia  indicators  of  several 
variables  converging  to  zero  have  similar  behavior  at  iteration  9.  Others  have  different  behavior  at 
iteration  10.  However,  it  is  not  clear  that  this  behavior  has  any  special  significance,  although  it 
may  suggest  that  certain  collections  of  variables,  either  primal  or  dual,  have  similar  behavior. 

2.1  The  Behavior  of  the  Tapia  Indicators  when  crk  — >  a  >  0 

The  behavior  of  the  Tapia  indicators  when  ak  — »  a  >  0  has  not  been  studied  so  far.  One  reason  for 
studying  such  behavior  is  that  in  order  to  converge  to  the  analytic  center  of  the  solution  set  S  the 
current  theory  requires  that  the  iteration  sequence  {(x^,  yk ,  Afc)}  converges  and  that  ak  be  bounded 
away  from  zero,  see  Zhang  and  Tapia  [32].  Another  reason  is  that  some  effective  implementation 
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ADLITTLE  sigma  — >  0 


Figure  1:  The  Tapia  primal  indicator  for  all  variables  in  ADLITTLE. 

of  interior-point  methods,  such  as  OBI  [18],  locally  use  a  positive,  yet  very  small,  constant  value 
for  ak. 

In  the  case  that  {<rfc}  is  chosen  to  be  bounded  away  from  zero,  El-Bakry,  Tapia  and  Zhang 
[5]  pointed  out  that,  with  the  choice  ak  —  min(l,  Tkak),  the  0-1  separation  property  of  the  Tapia 
indicator  cannot  be  retained  even  if  the  iteration  sequence  converges. 

In  an  attempt  to  study  the  behavior  of  the  Tapia  indicators  in  that  case  we  performed  numerical 
experimentation  with  the  Netlib  set  of  linear  programming  test  problems.  Details  of  the  numerical 
experiments  are  given  in  Section  4.  The  results  were  interesting  yet  their  intrepertation  was  quiet 
challenging  and  led  to  some  important  questions  concerning  the  behavior  of  the  iteration  sequence 
generated  by  Algorithm  1.  They  also  led  to  the  introduction  of  new  indicators  that  have  a  powerful 
ability  to  identify  several  groups  of  variables. 

We  first  start  by  showing  an  example  of  the  behavior  of  the  Tapia  indicators  for  problem 
ADLITTLE  when  ak  =  0.1.  Observe  that  the  convergence  of  the  Tapia  indicators  corresponding 
to  variables  that  are  positive  in  the  relative  interior  of  the  solution  set  is  somewhat  blured??,  see 
Figure  2.1.  Observe  also  that  the  dual  Tapia  indicators  do  not  exhibit  such  behavior,  see  Figure  2.1. 
In  other  words  the  Tapia  indicators,  for  variables  that  are  positive  in  the  realtive  interior  of  the 
solution  set,  do  not  seem  to  converge  to  1.  This  observation  is  more  evident  in  problem  AGG.  In 
this  problem  the  primal  logarithmic  Tapia  indicator  seem  actually  to  deviate  from  1  in  the  last  two 
iteration  as  seen  in  Figure  2.1.  The  behavior  can  be  quite  ??  as  seen  in  Figure  2.1.  The  dual  Tapia 
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Iteration  number 

Figure  2:  The  primal  Tapia  indicator  for  all  variables  in  ADLITTLE  with  crk  =  0.1. 

indicators  corresponding  to  a  subset  of  positive  variables  seem  to  approach  values  different  than 
1.  We  point  out  that  in  the  case  of  problem  AGG,  the  problem  was  solved  with  tighter  stopping 
criterion  than  that  used  in  ADLITTLE.  We  will  come  back  to  this  point  in  Section  4.  We  emphasize 
that  these  phenomena  appeared  in  many  probelms  in  the  Netlib  set. 

As  a  consequence  of  these  observations  it  was  evident  that  studying  these  phenomena  requires 
different  tools  than  the  existing  indicators  provide.  This  concern  was  the  motivation  for  introducing 
the  logarithmic  indicators. 

3  The  Logarithmic  Indicators 

In  this  section  we  introduce  several  logarithmic  indicators  constructed  from  the  three  existing 
indicators  discussed  in  Section  3.  These  indicators  are 

•  The  logarithmic  primal-dual  indicator 

LOGPD(xk,yk)  =  log(i), 

Vi 
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Figure  3:  The  dual  Tapia  indicator  for  all  variables  in  ADLITTLE  with  ak  =  0.1. 


•  The  logarithmic  variable  indicators 

LOGVP(xk )  =  log(xf), 

and 

LOGVD(xk)  =  log(2/f). 

•  The  logarithmic  Tapia  indicators 

LOGTP{xki)  =  log  |  1  -  TP(x\)  |, 

and 

LOGTD(yk)  =  log  |  1  -  TD(xk)  \  . 

The  separation  propertyies  for  these  indicators  can  be  easily  derived  for  the  case  when  ak  — ►  0 
at  least  R-linearly.  For  LOGVp  we  have 


lim  LOGVP(xk) 


— oo  if  i  6  Z 

log(x*)  if  i  £  Z 
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Iteration  number 


Figure  4:  The  primal  Tapia  indicator  for  all  variables  in  AGG  with  ak  =  0.1. 


where  x*  is  the  limit  point  of  {a;*"}  which  is  guaranteed  to  converge  by  Theorem  ??  of  Tapia,  Zhang, 
and  Ye  [26].  For  LOGPD  we  have 


lim  LOGPD(xk,yk ) 

k — ►  oo 


—  oo  if  i  £  Z 
+  oo  if  i  £  Z 


Here  we  used  the  convention  log(+oo)  =  +oo.  For  the  logarithmic  Tapia  indicators  we  have 


lim  LOGTP{x\ ) 


0  if  i  €  Z 
-oo  if  i  $  Z 


The  separation  property  for  the  logarithmic  primal-dual  indicator  is  the  same  if  the  sequence 
of  centering  paramaters  is  bounded  away  from  zero.  It  is  not  clear,  however,  what  would  be  the 
separation  property  for  the  logarithmic  Tapia  indicators,  if  there  is  any,  in  that  case.  Extensive 
numerical  experimentation  is  performed  in  the  next  section  to  study  the  logaritmic  Tapia  indicators 
when  ak  >  a  >  0. 
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Figure  5:  The  primal  Tapia  indicator  for  all  variables  in  AGG  with  ak  =  0.1. 


4  Numerical  Experiments 


In  this  section  we  present  several  numerical  experiments  with  the  logarithmic  indicators  introduced 
in  Section  3.  These  experiments  are  performed  on  a  subset  of  the  Netlib  test  set  using  a  predictor- 
corrector  primal-dual  interior-point  code  that  was  developed  at  Rice  University.  The  code  generates 
a  sequence  of  iterates  that  approach  feasibility  and  drive  the  duality  gap  cTx  -  bTy  to  zero.  For 

T  _ i  T 

numerical  purposes  our  stopping  criterion  is  stated  in  terms  of  the  relative  gap  rather  than 

the  gap.  We  will  say  that  a  problem  is  solved  to  an  accuracy  of  10-,i  for  some  positive  integer  d  if 
the  algorithm  is  terminated  when 


( 1  cTxk  -  bTyk\  || Axk  -  6||i  ||ATAfc  +  yk  -  c||i  \ 
\  l  +  \bTyk\  ’  1  +  ||x*||i  ’  1  +  ||Afc||i  +  ||yfc||1y 


<  €exit  =  I0~d. 


In  the  following  experiments  all  problems  are  solved  to  an  accuracy  of  10-8  unless  otherwise  spec¬ 
ified.  This  choice  agrees  with  the  default  choice  for  the  stopping  criterion  in  many  interior-point 
codes.  The  experiments  were  performed  on  a  Sun  4/490  workstation  with  64  Megabytes  of  memory. 
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4.1  The  Logarithmic  Tapia  Indicators 


The  behavior  of  the  logarithmic  Tapia  indicartors  was  quite  surprising.  Figure  4.1  shows  the 
behavior  of  the  dual  logarithmic  Tapia  indicator  when  ak  =  0.1  for  all  k  for  problem  LOTFI.  The 


Iteraion  number 

Figure  1:  The  dual  logarithmic  Tapia  indicator  for  all  variables  in  LOTFI  with  ak  —*  0.1. 

trajectories  of  the  dual  logarithmic  Tapia  indicators  split  into  four  groups  instead  of  the  two  basic 
groups  (variables  that  are  zero  everywhere  on  the  solution  set  and  variables  that  are  positive  in  the 
relative  interior  of  the  solution  set).  So  it  seems  that  the  logarithmic  Tapia  indicators  are  able  to 
identify  several  subgroups  of  variables  instead  of  only  the  two  basic  groups. 

In  the  following  we  try  to  explain  this  behavior. 

4.1.1  Variables  Invariant  on  the  Solution  Set 

El-Bakry,  Tapia  and  Zhang  [5]  pointed  out  that  the  convergence  of  the  Tapia  indicators  does  not 
require  the  convergence  of  the  iteration  sequence,  but  only  the  milder  condition  that  the  sequence 
{(Aa^,  A^)}  converges  to  zero.  For  Algorithm  1,  the  current  theory  (see  Tapia,  Zhang,  and  Ye  [26]) 
ensures  that  this  will  occur  when  ak  — >  0.  In  this  case  the  primal  Tapia  indicator  corresponding 
to  variables  that  are  positive  in  the  relative  interior  of  the  solution  set  converges  to  1  (even  if  the 
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iteration  sequence  itself  does  not  converge).  This  in  turn  implies  that  the  logaritmic  Tapia  indicator 
corresponding  to  such  variables  approachs  infinity  in  the  limit.  Observe  that  the  logarithmic  Tapia 
indicators  corresponding  to  some  variables  seem  to  approachinfinity  in  Figure  4.1.  Several  of  these 
indicators  do  so  very  early  (at  iteration  4)  while  others  approach  infinity  only  when  the  iterates 
are  close  to  the  solution  set.  On  the  other  hand  observe  that  there  is  a  third  group  that  does  not 
seem  to  approach  infinity  at  all.  To  check  this  phenomenon  in  other  problems  we  ran  the  code 
with  ak  —  0.1  on  problem  AFIRO.  The  same  phenomenon  occured  again,  see  Figure  4.1.1.  We  see 
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Figure  2:  The  logarithmic  Tapia  primal  indicator  for  all  variables  in  AFIRO  with  ak  =  0.1. 

that  there  is  a  group  of  variables  with  their  corresponding  logarithmic  Tapia  indicator  approaching 
some  intermediate  value. 

Solving  the  same  problem  (AFIRO)  with  ak  =  0({xk)Tyk)  and  also  by  CPLEX  [].  Comparing 
the  three  approximate  solutions  we  observed  the  following,  see  Table  (4.1.1) 

(1)  Variables  with  the  corresponding  primal  logarithmic  Tapia  indicator  approaching  values  in 
the  range  (  —  7,  —4)  are  different  in  both  solutions. 

(2)  Variables  with  the  corresponding  primal  logarithmic  Tapia  indicator  approaching  -oo  have 
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the  same  value  (to  at  least  eight  significant  digits)  in  the  three  approximate  solutions.  More¬ 
over  those  variables  are  basic  in  the  CEPLEX  solution. 


Variable  Name 

Xj 

Xj 

X  j  X  j 

Status 

X01 

7.99999999e+01 

8.00000000e+01 

-9.9999994e-08 

basic 

X02 

2.54999999e+01 

2.55000000e+01 

-1.0000000e-07 

basic 

X03 

5.45000000e+01 

5.45000000e+01 

0 

basic 

X04 

8.47999999e+01 

8.48000000e+01 

-9.9999994e-08 

basic 

X06 

6.70725826e+01 

7.04887640e+01 

-3.4161813e+00 

basic 

X07 

1.82142856e+01 

1.82142857e+01 

-1.0000000e-07 

basic 

X15 

4.88582971e+01 

5.22744784e+01 

-3.4161812e+00 

nonbasic 

X16 

7.10969376e+01 

7.47180899e+01 

-3.6211522e+00 

basic 

X17 

1.29274174e+01 

9.51123596e+00 

3.41618143e+00 

basic 

X22 

5.00000000e+02 

5.00000000e+02 

0 

basic 

X23 

4.75919999e+02 

4.75920000e+02 

-9.9999999e-07 

basic 

X24 

2.40800000e+01 

2.40800000e+01 

0 

basic 

X26 

2.15000000e+02 

2.15000000e+02 

0 

basic 

X28 

1.39125795e+02 

2.71341050e+02 

-1.3221525e+02 

nonbasic 

X36 

3.39942857e+02 

3.39942857e+02 

0 

basic 

X37 

2.44817061e+02 

1.12601807e+02 

1.32215254e+02 

basic 

X38 

5.98240921e+01 

1.16676651e+02 

-5.6852558e+01 

basic 

X40 

3.60874205e+02 

2.28658950e+02 

1.32215255e+02 

basic 

X47 

3.36935854e+01 

2.26983040e+01 

1.09952814e+01 

X49 

2.24628214e+02 

9.13846893e+01 

1.33243524e+02 

X50 

1. 0200000  le+01 

1.02000000e+01 

1.00000001e-07 

X51 

1.69078970e+02 

1.08605259e+02 

6.04737109e+01 

basic 

Table  1:  All  positive  variables  in  the  approximate  solution  for  problem  AFIRO 


This  suggests  that  the  logarithmic  Tapia  indicators  have  the  ability  to  distinguish  between  positive 
variables  that  are  invariant  throughout  the  solution  set  and  those  whose  values  change  from  solution 
to  solution. 


4.1.2  Variables  Invariant  on  the  Feasible  Set 

We  now  direct  out  attention  back  to  Figure  4.1  where  the  dual  logarithmic  Tapia  indicator  is  plotted 
against  the  iteration  number.  Again  the  previously  discussed  subgroups  of  variables  in  the  three 
categories  of  variables  that  are  zero  throught  the  solution  set,  positive  variables  that  are  invariant 
on  the  solution  set,  and  positive  variables  that  have  different  values  in  different  solutions  are  clearly 
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shown. 


A  new  subgroup  of  variables  manifests  itself  with  the  property  that  they  reach  their  terminal 
values  very  early  in  the  course  of  the  algorithm.  This  is  reflected  in  the  behavior  of  the  corresponding 
dual  logarithmic  Tapia  indicator  approaching  — oo  from  iteration  4  and  settling  down  there.  The 
same  phenomenon  occured  again  for  SHARE1B,  but  at  a  relatively  later  stage  (at  iteration  12), 
see  Figure  4.1.2.  We  observed  that  at  iteration  12  the  relative  primal  feasibility  residual  became 
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Figure  3:  The  logarithmic  Tapia  dual  indicator  for  all  variables  in  SHARE1B  with  ak  — ►  0.1. 

2.0  X  10~12  after  being  1.0  X  10-2  at  iteration  11.  So  a  plausible  interpertation  for  the  behavior  of 
the  logarithmic  Tapia  indicator  in  this  case  is  that  the  logarithmic  Tapia  indicator  is  able  to  identify 
positive  variables  that  are  invariant  (have  the  same  value  )  throughout  the  primal  feasibility  region 
{x  :  Ax  —  b  —  0,x  >  0}  when  the  iterate  xk  is  sufficiently  close  to  {x  :  Ax  -  b  =  0,x  >  0}.  It  is 
also  interesting  to  observe  that  dual  feasibility  is  not  needed  for  such  identification  (relative  dual 
feasibility  for  SHARE1B  was  of  order  10~3  at  iterations  11,  12,  and  13). 
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4.1.3  Variables  Unbounded  on  the  Solution  Set 


Solving  problem  ADLITTLE  with  ak  =  0.1,  we  observed  that  the  code  terminated  with  relative 
feasibility  of  order  10~8  while  the  dual  feasibility  HA^A  +  y  —  c||,  where  (x,  A,  y )  is  the  approximate 
solution  given  by  the  code,  was  15.6,  which  is  not  small.  One  explanation  is  that  (x,  A,  y )  is  not  close 
enough  to  the  solution  set.  To  check  if  this  is  the  case,  we  resolved  the  problem  to  accuracies  10~10 
and  10~13  asking  for  increasingly  more  accurate  approximate  solutions.  The  results  are  shown  in 
Table  (4.1.3).  In  all  cases  ||ATA  +  y  —  c||  is  large  although  the  solution  seems  more  accurate  (which 


^ exit 

relative 

gap 

relative  primal 
feasibility 

relative  dual 

feasibility 

10“8 

1.4D-09 

6.1D-15 

3.1D-18 

15.6 

io-10 

1.1D-11 

6.0D-15 

1.0D-30 

15.6 

10-13 

9.1D-14 

2.0D-15 

3.2D-47 

15.6 

is  reflected  by  the  decrease  in  the  relative  gap)  as  we  reduced  eexa . 

Another  explanation  is  that  some  of  the  dual  variables  are  unbounded  on  the  solution  set  at 
the  approximate  solution  given  by  the  code.  Both  primal  and  dual  logarithmic  Tapia  indicators 
are  computed  and  plotted  in  Figures  4.1.3  and  4.1.3  respectively.  First  we  note  that  the  three 
groups  of  variables  discussed  earlier  for  AFIRO  appear  in  Figure  4.1.3.  However  observe  that  in 
Figure  4.1.3,  one  of  the  trajectories  seems  to  approach  +oo.  This  suggests  that  the  corresponding 
dual  slack  is  also  approaching  +oo,  and  hence  unbounded  in  the  solution  set.  The  value  of  this 
variable  at  x,  the  approximate  solution  given  by  the  code,  is  2.06  X  1019. 

To  investigate  the  validity  of  such  a  claim,  we  solved  the  same  problem  with  smaller  values  of  a , 
decreasing  a  an  order  of  magnitude  each  time.  For  a  =  0.01, 0.001,  we  obtain  similar  behavior.  For 
smaller  values  of  a  the  dual  logarithmic  Tapia  indicator,  corresponding  to  the  same  variable,  does 
not  seem  to  approach  infinity  although  it  still  has  a  very  peculiar  behavior  (see  Figure  4.1.3)  for 
a  =  10~5.  Finally,  it  is  intersting  to  know  that  if  we  solve  the  problem  with  ak  =  0((xk)Tyk),  the 
dual  logarithmic  Tapia  indicator  behavior  becomes  similar  to  any  other  variable,  see  Figure  4.1.3. 
We  attempt  to  explain  this  behavior  in  Section  5. 

4.1.4  Yet  Another  Subgroup  of  Variables 

Another  subgroup  of  variables  was  discovered  in  some  problems  of  the  Netlib  set.  When  we  solved 
problem  AGG,  again  with  ak  —  0.1,  the  behavior  of  the  primal  logarithmic  Tapia  indicator  for  some 
variables  was  peculiar,  see  Figure  4.1.4.  The  trajectories  for  several  logarithmic  Tapia  indicators 
was  oscillating  between,  seemingly,  two  different  values  as  the  corresponding  variable  approached  its 
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Figure  4:  The  primal  logarithmic  Tapia  indicator  for  all  variables  in  ADLITTLE  with  ak  =  0.1. 

terminal  value.  The  corresponding  primal  variable  is  zero  at  the  approximate  solution  obtained  by 
the  code.  The  corresponding  dual  variable  has  the  large  value  of  10s,  but  its  dual  logarithmic  Tapia 
indicator  does  not  approach  -foo.  This  behavior  was  observed  in  several  problems,  e.g.  SHIP04S 
and  FORPLAN.  The  interpertation  of  such  behavior  is  still  an  open  question. 

4.2  Other  Logarithmic  Indicators 

In  this  section  we  consider  the  logarithmic  variable  indicators  and  the  logarithmic  primal-dual  in¬ 
dicator.  Although  these  indicators  have  the  ability  to  identify  more  subgroups  of  variables,  see  for 
example  Figures  4.2  and  4.2,  than  either  the  primal-dual  indicator  or  the  variables  as  indicators, 
they  cannot  compete  with  the  logarithmic  Tapia  indicators  in  identifying  all  the  subgroups  that 
the  logaritmic  Tapia  indicator  identifies.  For  example  the  primal-dual  indicator  did  demonstrate 
a  strong  ability  to  identify  members  in  Tf ,  see  Figure  4.2  for  primal-dual  indicators  for  problem 
SHARE1B  and  compare  to  the  corresponding  Figure  4.1.2  for  the  primal  logarithmic  Tapia  indi¬ 
cator.  The  reason  is  simple.  Although  variables  in  approach  their  terminal  values  once  primal 
feasibility  is  reached,  the  corresponding  dual  variables,  whose  terminal  values  are  zero,  are  not  suf- 
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Figure  5:  The  primal  logarithmic  Tapia  indicator  for  all  variables  in  ADLITTLE  with  crk  =  0.1. 

ficiently  close  to  zero.  Moreover  both  logarithmic  primal-dual  and  logarithmic  variable  indicators 
were  not  able  to  identify  variables  in  Vc- 

5  Interpretation  of  the  Logarithmic  Tapia  Indicators  Behavior 
when  a  >  0 

Although  the  surprising  ability  of  the  logarithmic  Tapia  indicator  to  identify  several  subgroups  of 
variables  is  impressive,  an  attempt  to  explain  thier  difference  was  quite  challenging.  We  first  define 
the  following  sets 

•  Vp  =  {x  :  Ax  —  b,x  >  0}  denotes  the  primal  feasible  region. 

•  Sp  denotes  the  primal  solution  set. 

•  Vc  =  {i  :  =  fit  where  /;  is  constant  for  all  x  G  «$P}- 

•  Vnc  =  {*  :  X{  =  fi(x )  >  0,  where  fi(x)  is  not  constant  for  x  £  Sp}. 
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Figure  6:  The  logarithmic  dual  Tapia  indicator  for  all  variables  in  ADLITTLE  with  ak  =  10~5. 


•  'Pri  =  {*  :  Xi  =  fi(x)  >  0,  where  fi(x)  is  not  constant  and  fi(x)  =  0  on  dTSp} 

•  Pub  =  {i  :  Xi  >  M  for  any  positive  constant  M  and  for  at  least  one  x  £  sO- 

•  Tc  —  {i  :  xi  =  fi,  where  c;  is  constant  for  all  a;  £  Tp}. 

We  define  the  corresponding  sets  for  the  dual  slacks  z  similarly. 

It  is  easy  to  establish  the  separation  property  for  the  logarithmic  Tapia  indicators  as  follows 


Proposition  5.1  Let  the  sequence  of  iterates  {(xk,yk,  Afc)}  be  generated  by  Algorithm  1.  Assume 
that 

1.  S  is  bounded. 

2.  (xk)Tyk  -*•  0. 

rkyk 


3-  "7^  >  7  for  all  k  and  some  positive  constant  7. 

4 ■  crk  — *■  0  and  rk  — *  1. 
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Figure  7:  The  dual  logarithmic  Tapia  indicator  for  all  variables  in  ADLITTLE  with  ok  — ►  0.1. 


Then  for  i  =  1, . . . ,  n 


lim  LOGTP(xk ) 


0  if  i  €  Z 
-oo  if  i  £  Z 


Proof:  The  proof  is  straightforward.  □ 

We  note  here  that  Conditions  1-4  of  Proposition  5.1  ensure  the  convergence  of  the  step  ( Axk ,  A yk) 
to  zero,  and  this  is  what  is  needed  to  ensure  the  properties  of  the  Tapia  (or  the  logarithmic  Tapia) 
indicators,  see  El-Bakry,  Tapia  and  Zhang  [5].  If  condition  (iii)  is  slightly  strengthened  in  the 
sense  that  ak  is  required  to  converge  to  zero  at  least  R-linearly  then  these  assumption  ensures  the 
convergence  of  the  iteration  sequence  (xk,yk)  to  a  solution  (x*,y*)  €  ri(S),  see  Zhang,  Tapia,  and 
Ye  [26], 

The  separation  property  of  the  logarithmic  Tapia  indicator  is  established  in  the  following  propo¬ 
sition. 
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Figure  8:  The  primal  logarithmic  Tapia  indicator  for  all  variables  in  AGG  with  ok  — >  0.1. 

5.1  Behavior  of  the  Logarithmic  Tapia  Indicators  when  ak  =  a  >  0 

First  we  establish  the  separation  property  of  the  logarithmic  Tapia  indicator  corresponding  to  the 
two  basic  groups  of  variables. 

Proposition  5.2  Let  the  sequence  of  iterates  {(xk,yk,Xk)}  be  generated  by  Algorithm  1.  Assume 
that 

1.  S  is  bounded. 

2.  (xk,yk)  — ►  (x*,y*). 

3.  m,^kjTyk  ^  7  for  k  and  some  positive  constant  7 . 

4-  Ok  =  <T  >  0. 

5.  ak  — >  a  G  (0, 1]. 

Then 

00 


if  i  £  Z 
if  i  £  Z 


(5.1) 


lim  LOGTp(xk) 
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Figure  9:  The  dual  logarithmic  Tapia  indicator  for  all  variables  in  AGG  with  ok  — ►  0.1. 
where  rj  —  log(l  —  a(l  —  a)). 

Note  that  the  convergence  of  the  iteration  sequence,  generated  by  Algorithm  1,  when  ak  >  a  >  0, 
has  not  been  established.  We  also  note  that  it  is  not  clear  that  ak  — ►  a  €  (0, 1)  under  Condition  4 
of  Proposition  5.2. 

5.2  Variables  in  Z  U  Vc 

We  first  consider  the  following  proposition  that  slightly  extends  conclusion  (i)  of  Theorem  (4.1)  in 
Tapia,  Zhang  and  Ye  [26].  It  basically  says  that  if  {(xk,yk)}  is  generated  by  Algorithm  1,  then  the 
variables  with  indices  in  Z  U  Vc,  i.e.  zero  variables  and  variables  that  are  invariant  throughtout  the 
solution  set,  converge.  The  proof  is  straightforward  and  will  be  omitted. 

Proposition  5.3  Let  the  sequence  {(xk,  yk,  \k)}  be  generated  by  Algorithm  1.  Assume  further  that 

1.  ( xk)Tyk  ->  0. 

2.  min(XkYke)/(xk)Tyk  >  ^  for  all  k  and  some  positive  7. 
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Figure  10:  The  primal  logarithmic  variable  indicator  for  all  variables  in  SHARE1B  with  rrk  —?  0.1. 

Then  for  all  i  €  Z  U  Vc,  the  sequence  {m* }  converges  to  x* . 

This  proposition  explains  the  behavior  of  the  logarithmic  Tapia  indicator  for  variables  with 
i  £  Vc-  For  any  of  these  variables  the  sequence  {xk}  converges  and  hence 

xk+1 

1  k  — *  1,  which  implies  that  LOGTp(xk)  — >  — oo.  (5.2) 

Xi 

For  a  variable  Xj  with  j  €  Z,  assuming  that  the  iteration  sequence  {yk }  converges  to  yf,  we  have 

LOGT{xki )  log(  1  -  a(l  -  a)).  (5.3) 

Note  that  we  assumed  that  the  dual  variable  sequence  {yk }  converges.  This  is  guaranteed  if  y*  has  a 
constant  value  in  all  dual  solutions  (a  trivial  special  case  is  the  uniqueness  of  dual  solution).  Hence 
in  these  two  cases  the  primal  logarithmic  Tapia  indicator  behavior  is  explained.  In  the  following 
we  try  to  explain  their  behavior  for  other  subgroups  of  variables. 
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Figure  11:  The  logarithmic  primal-dual  indicator  for  all  variables  in  SHARE1B  with  ak  — ►  0.1. 

5.3  Variables  in  V  —  Vc 

For  variables  that  have  different  values  at  different  solutions  of  problem  (1.1),  we  give  two  plausible 
explanations  under  different  assumptions. 

First  assume  that 

(Al)  a*  >  a  >  0  and  ak  >  a  for  some  a,a  £  (0, 1). 

( A2)  the  iteration  sequence  {( xk ,  yk,  A*1)}  generated  by  Algorithm  1  converges  to  a  point  ( x *,  y*,  A*) 
in  the  realtive  interior  of  the  solution  set. 

By  Theorem  2.1  of  Zhang  and  Tapia  [32]  the  limit  point  is  the  analytic  center  of  the  solution  set. 

Without  loss  of  generality,  consider  an  arbitrary  trajectory  of  the  primal  logarithmic  Tapia 
indicator  in  any  of  the  previous  figures  for  the  case  ak  —  a  =  0.1.  For  k  sufficiently  large  this 
trajectory  closely  approximates  a  straight  line.  Observe  that  the  slope  of  these  straight  lines  is 
the  same  for  variables  in  Vc  and  variables  in  Vnc.  For  all  examples  that  we  have  in  the  previous 
sections  the  slope  was  nonpositive.  So  assume  that  the  slope  for  a  given  curve  is  — where  £,•  >  0. 
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larger  than  the  Qi  factor  for  variables  in  Vc.  This  implies  that  {Axf}  for  i  £  V  -  Vc  converges  to 
zero  slower  than  {Azf}  for  i  6  Vc- 

On  the  other  hand,  if  the  iteration  sequence  {xi}iev-v  c  does  not  converge  then  xk+1  /xk  ■/+  1 
which  results  in  LOGTp(xk )  — oo.  So  our  second  interpertation  suggests  that  the  iteration 

sequence  generated  by  Algorithm  1  may  not  actually  converge. 

The  current  convergence  theory  for  the  iteration  sequence  includes  Gonzaga  and  Tapia  [10]  and 
[11]  and  Tapia,  Zhang  and  Ye  [26].  Gonzaga  and  Tapia  [10]  consider  convergence  of  a  modification 
of  the  Mizuno-Todd-Ye  algorithm  which  does  not  fit  in  the  framework  considered  in  this  paper. 
Tapia,  Zhang  and  Ye  [26]  considered  the  iteration  sequence  generated  by  Algorithm  1  and  proved 
that  the  sequence  converges  under  the  assumption,  in  addition  to  other  assumptions,  that  {crk} 
converges  to  zero  at  least  R-linearly.  Although  it  is  widely  believed  that  the  sequence  converges  for 
ak  >  a  >  0,  this  has  not  been  proved  or  disproved. 


5.4  Variables  in  Vub 


If,  for  some  i  €  Af,  we  have 


LOGTp(x^) 


Too  then 


+  oo, 


(5.4) 


which  indicates  that  the  sequence  {a:^}  is  unbounded.  This  in  turn,  since  that  sequence  is  approach¬ 
ing  the  solution  set,  indicates  that  the  corresponding  variable  X{  is  unbounded  on  the  solution  set 
Sp .  Note  also  that  (5.4)  implies  that 


a 


k 


Axk 

xk 


Too  and  hence 


Too, 


since  xk  is  bounded  away  from  zero  and  ak  <  1  from  Algorithm  1.  For  an  analysis  of  the  limiting 
behavior  of  infeasible  interior-point  algorithms  in  the  presence  of  unbounded  variables,  see  Mizuno, 
Todd,  and  Ye  [22]. 

To  explain  why  this  phenomenon  is  not  encountered  when  ak  is  chosen  so  that  ak  — ►  0,  we 
recall  the  step  bounded  deterioration  property  of  Tapia,  Zhang  and  Ye  [26], 


\Axl?\<p1(xk)Tyk+(32ak, 


for  some  positive  constants  and  /?2-  So  it  is  clear  that,  under  the  assumption  that  ( xk)Tyk  — >  0, 
the  step  Axk  converges  to  zero  as  long  as  <rk  — >  0.  This  demonstrates  that  if  we  choose  crk  — ►  0, 
the  perturbed  Newton  step  is  forced  to  zero,  and  the  volatile  behavior  of  unbounded  variables  is 
suppressed,  see  Figure  5. 
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5.5  Variables  in  Tc 


Assume  that  the  sequence  {£j}  generated  by  Algorithm  1  is  bounded.  Assume  further  that  the 
feasible  set  Tp  of  problem  (1.1)  is  closed  and  bounded.  If  Xj  is  invariant  throughout  Tp  for  a  given 
j,  then  {xk}  converges  to  xj  as  the  iterates  ( xk,yk,Xk )  approach  primal  feasibihty.  In  that  case 


1  as  Axk  -  b 


0, 


which  explains  the  behavior  of  the  Tapia  indicator  in  this  case. 


6  Dual  Variables 

The  behavior  of  the  Tapia  indictor  depends  essentially  on  three  factors,  (i)  the  limit  points  of 
{(xfc,  zk)}  satisfy  strict  complemetarity,  (ii)  linearized  complementarity  is  approached,  and  (iii) 
the  sequence  of  steps  {(Axk,  A.^)}  converges  to  zero.  However  a  surprising  phenomenon  was 
observed  when  the  logarithmic  Tapia  indicators  were  applied  to  the  sequence  of  dual  variables  {yk} 
(Lagrange  multipliers  corresponding  to  the  equality  constraints  Ax  —  b  =  0).  The  Tapia  indicator 
trajectories  seem  to  cluster  near  two  values,  with  the  corresponding  variables  being  zero  or  positive 
on  the  approximate  solution  obtained  from  the  code.  Moreover,  the  logarithmic  Tapia  indicator 
trajectories  have  the  same  behavior  for  the  logarithmic  Tapia  indicators  for  either  {xfc}  or  {zk}. 
The  trajectories  split  into  several  groups.  Two  examples  are  given  in  Figure  6  for  problems  BLEND 
and  SCSD1.  It  is  worth  mentioning  that  all  groups  of  logarithmic  Tapia  indicators  discussed  in 
Section  4  were  observed  for  the  dual  variables  y. 

The  behavior  of  the  logarithmic  Tapia  indicators  corresponding  to  variables  yj  with  y*  >  0  is 
no  surprise  if  the  sequence  { yk }  converges.  But  the  split  between  the  trajectories  corresponding  to 
positive  variables  as  well  as  the  clustering  of  the  ones  corresponding  to  zero  dual  variables  near  a  is 
not  expected.  The  reason  being  that  these  variables  do  not  satisfy  any  complementarity  conditions. 

We  point  out  that  if  some  of  the  primal  variables  x  are  added  as  slack  variables  to  transform 
the  primal  problem  into  the  form  (1.1),  then  some  of  the  dual  variables  yj  will  be  related  to  a  dual 
slack  Z£  by  the  following  relation 

yj  +  z(  =  ct, 

which  may  explain  the  behavior  of  the  Tapia  (and  logarithmic  Tapia  indicator)  for  the  dual  variables 
y.  This  suggests  that  the  behavior  of  the  dual  variables  y  plays  a  nontrivial  role  in  the  behavior  of 
the  primal-dual  interior- point  methods. 
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Figure  13:  The  logarithmic  Tapia  indicator  for  all  dual  variables  in  BLEND  and  SCSD1. 

7  Conclusion 

In  this  paper,  several  logarithmic  indicators  have  been  introduced.  Among  these  the  logaritmic 
Tapia  indicators  have  proven  to  be  powerful  tools  for  identifying  several  subgroups  of  variables 
in  interior-point  methods.  We  have  not  been  able  to  obtain  a  definite  intrepertation  for  their 
behavior  for  some  subgroups  of  variables.  We  believe  that,  with  the  understanding  gained  from 
these  indicators  on  the  behavior  of  subgroups  of  variables,  more  efficient  implementation  of  finite 
termination  and  hybrid  methods  is  possible.  We  also  believe  that  the  behavior  of  several  groups  of 
variables  merits  further  study  to  analyze  its  effect  on  the  implementation  as  well  as  the  theory  of 
interior-point  methods. 
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